🪞 [MIRRORED] fix: Annotate PLR when when started status is reported #2209
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Summary of Changes
Hello @zakisk, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!
This pull request introduces a mechanism to precisely track and report the "started" status of PipelineRuns to Source Code Management (SCM) providers. Previously, there might have been issues with reporting the "in progress" status multiple times or incorrectly. My changes ensure that the SCM provider is notified exactly once when a PipelineRun transitions from a queued state to a running state, improving the accuracy and reliability of status updates.
Highlights
- Dedicated Annotation for SCM Reporting: I've introduced a new annotation key, SCMReportingPLRStarted, to explicitly mark when a PipelineRun's "started" status has been reported to the SCM provider. This prevents redundant or incorrect status updates.
- Refined PipelineRun Status Logic: The logic for updating PipelineRun states and reporting them to SCM providers has been enhanced. Now, the system checks this new annotation before sending an "in progress" status, ensuring it's only sent once per PipelineRun.
- Improved Reconciliation Robustness: I've added a step in the reconciler to fetch the latest PipelineRun object directly from the API server. This mitigates potential issues caused by stale informer caches, ensuring that reconciliation decisions are always based on the most current state.
- Enhanced Observability: I've added more detailed logging to provide better insights into the PipelineRun patching process, including the values of the new SCMReportingPLRStarted annotation.
- Comprehensive Test Coverage: New unit tests have been added to thoroughly validate the behavior of the new SCMReportingPLRStarted annotation and the updated reconciliation logic, ensuring the fix works as expected.
Using Gemini Code Assist
The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.
Invoking Gemini
You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.
| Feature | Command | Description |
|---|---|---|
| Code Review | /gemini review |
Performs a code review for the current pull request in its current state. |
| Pull Request Summary | /gemini summary |
Provides a summary of the current pull request in its current state. |
| Comment | @gemini-code-assist | Responds in comments when explicitly tagged, both in issue comments and review comments. |
| Help | /gemini help |
Displays a list of available commands. |
Customization
To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.
Limitations & Feedback
Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments or fill out our survey to provide feedback.
You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.
Footnotes
-
Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩
There was a problem hiding this comment.
Code Review
This pull request introduces a new annotation, SCMReportingPLRStarted, to track whether a PipelineRun's 'started' status has been reported to the SCM. This prevents duplicate status updates and makes the reconciliation logic more robust. The changes are well-implemented across the codebase, including updates to the reconciler, the main pipeline-as-code logic, and corresponding tests.
My review identifies one issue in a new test case where a necessary field was not initialized, which could lead to a panic. I've provided a fix for this.
| r := &Reconciler{ | ||
| repoLister: informers.Repository.Lister(), | ||
| run: ¶ms.Run{ | ||
| Clients: clients.Clients{ | ||
| Tekton: stdata.Pipeline, | ||
| }, | ||
| Info: info.Info{ | ||
| Pac: &info.PacOpts{ | ||
| Settings: settings.Settings{}, | ||
| }, | ||
| }, | ||
| }, | ||
| } |
There was a problem hiding this comment.
The Reconciler instance in this test is missing the initialization for the kinteract field. This can lead to a nil pointer dereference when ReconcileKind calls updatePipelineRunToInProgress, which in turn uses kinteract to interact with secrets.
The run.Clients.Kube field also needs to be initialized.
Please update the Reconciler initialization to include kinteract and the Kubernetes client, like this:
r := &Reconciler{
repoLister: informers.Repository.Lister(),
run: ¶ms.Run{
Clients: clients.Clients{
Tekton: stdata.Pipeline,
Kube: stdata.Kube,
},
Info: info.Info{
Pac: &info.PacOpts{
Settings: settings.Settings{},
},
},
},
kinteract: &kubeinteraction.Interaction{
Run: ¶ms.Run{
Clients: clients.Clients{
Kube: stdata.Kube,
Tekton: stdata.Pipeline,
},
},
},
}This will make the test more robust and prevent potential panics.
The use of the state label (which is mutable) for deciding when to report to the SCM that the PLR was started is flaky. It was seen that the reconciler get events about PLRs with unexpected value for the state label. For example, after the status is reported to the SCM, and the label value is patched to "started", after serval reconcile iterations the label had the "queued" value again. This can happen because of unsafe patching done by controllers (not just the PAC controllers) which reconciles PLRs. Introduce a new annotation for indicating the the status was reported to the SCM. By adding an annotation which is set once, we remove the risk that its value will get overwritten by other controllers (since maps are merged when patched, values are not getting removed unless explicitly defined in the patch - https://datatracker.ietf.org/doc/html/rfc7386#section-2) In addition, at the start of each reconcile loop, ensure that we operate on the latest version of the PLR and not using a stale value from the cache. Assisted-By: Cursor Signed-off-by: Gal Ben Haim <[email protected]>
Set the state annotation and labels in pipelineascode only after the state was reported to the SCM. This is achieved by patching the PLR with the state label/annotation after it was created. This change is needed in order to avoid a case where the watcher will report a status before PAC. With this change the issue can't happen, since the watcher only reconciles PLRs that has the state annotation, so the first reconcile of the PLRs will happen only after PAC already sent the initial status to the SCM. In addition, simplify the condition which checks if the "queued" state annotation/label should be set. We can only check if the PLR is pending, it doesn't matter if the reason is because of PAC's concurrency control or an external controller. Signed-off-by: Gal Ben Haim <[email protected]>
zakisk
force-pushed
the
test-pr-2208-gbenhaim
branch
from
56793cb to
25bfed7
Compare
Instead of running the reconcile loop on the fresh pipeline, skip it and let the next iterations to reconcile it. Signed-off-by: Gal Ben Haim <[email protected]>
🔄 Mirrors #2208 to run E2E tests. Original author: @gbenhaim